SDTR: Soft Decision Tree Regressor for Tabular Data

Deep neural networks have been proved a success in multiple fields. However, researchers still favor traditional approaches to obtain more interpretable models, such as Bayesian methods and decision trees when processing heterogeneous tabular data. Such models are hard differentiate, thus inconvenient be integrated into end-to-end settings. On the other hand, differentiable but perform poorly on We propose hierarchical regression model, Soft Decision Tree Regressor (SDTR). SDTR imitates binary tree by network is plausible for ensemble schemes like bagging boosting. The method was evaluated tabular-based tasks (YearPredictionMSD, MSLR-Web10K, Yahoo LETOR, SARCOS Wine quality). Its performance comparable with non-differentiable (gradient boosting trees) better than uninterpretable (regular FCNN). top of that, it can produce fair results restricted number parameters, only using small forest or even single tree. also an “average entropy” metric evaluate level interpretability trained, soft network. This helps select proper structure hyperparameters networks.